<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "https://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
<meta http-equiv="Content-Type" content="text/xhtml;charset=UTF-8"/>
<meta http-equiv="X-UA-Compatible" content="IE=11"/>
<meta name="generator" content="Doxygen 1.9.4"/>
<meta name="viewport" content="width=device-width, initial-scale=1"/>
<title>Flow-IPC: Structured Message Transport: Messages As Long-lived Data Structures / Advanced Topics</title>
<link href="tabs.css" rel="stylesheet" type="text/css"/>
<script type="text/javascript" src="jquery.js"></script>
<script type="text/javascript" src="dynsections.js"></script>
<link href="search/search.css" rel="stylesheet" type="text/css"/>
<script type="text/javascript" src="search/searchdata.js"></script>
<script type="text/javascript" src="search/search.js"></script>
<link href="doxygen.css" rel="stylesheet" type="text/css" />
</head>
<body>
<div id="top"><!-- do not remove this div, it is closed by doxygen! -->
<div id="titlearea">
<table cellspacing="0" cellpadding="0">
 <tbody>
 <tr id="projectrow">
  <td id="projectalign">
   <div id="projectname">Flow-IPC<span id="projectnumber">&#160;1.0.2</span>
   </div>
   <div id="projectbrief">Flow-IPC project: Full implementation reference.</div>
  </td>
 </tr>
 </tbody>
</table>
</div>
<!-- end header part -->
<!-- Generated by Doxygen 1.9.4 -->
<script type="text/javascript">
/* @license magnet:?xt=urn:btih:d3d9a9a6595521f9666a5e94cc830dab83b65699&amp;dn=expat.txt MIT */
var searchBox = new SearchBox("searchBox", "search",'Search','.html');
/* @license-end */
</script>
<script type="text/javascript" src="menudata.js"></script>
<script type="text/javascript" src="menu.js"></script>
<script type="text/javascript">
/* @license magnet:?xt=urn:btih:d3d9a9a6595521f9666a5e94cc830dab83b65699&amp;dn=expat.txt MIT */
$(function() {
  initMenu('',true,false,'search.php','Search');
  $(document).ready(function() { init_search(); });
});
/* @license-end */
</script>
<div id="main-nav"></div>
<!-- window showing the filter options -->
<div id="MSearchSelectWindow"
     onmouseover="return searchBox.OnSearchSelectShow()"
     onmouseout="return searchBox.OnSearchSelectHide()"
     onkeydown="return searchBox.OnSearchSelectKey(event)">
</div>

<!-- iframe showing the search results (closed by default) -->
<div id="MSearchResultsWindow">
<iframe src="javascript:void(0)" frameborder="0" 
        name="MSearchResults" id="MSearchResults">
</iframe>
</div>

</div><!-- top -->
<div><div class="header">
  <div class="headertitle"><div class="title">Structured Message Transport: Messages As Long-lived Data Structures / Advanced Topics </div></div>
</div><!--header-->
<div class="contents">
<div class="textblock"><center><b>MANUAL NAVIGATION:</b> <a class="el" href="chan_struct.html">Preceding Page</a> - <a class="el" href="transport_shm.html">Next Page</a> - <a href="./pages.html"><b>Table of Contents</b></a> - <a class="el" href="namespaceipc.html"><b>Reference</b></a></center><hr  />
<p >We previously discussed how to transmit structured messages; in this page we get into advanced techniques that treat messages as long-lived data structures to re-share multiple times and potentially modify throughout. (Or go back to the prerequisite preceding page: <a class="el" href="chan_struct.html">Structured Message Transport</a>.)</p>
<h2><code>struc::Channel</code> and serialization </h2>
<p >In <a class="el" href="chan_struct.html">Structured Message Transport</a> we kept things simple: Create <code>struc::Channel</code>. Create a message <code>struc::Msg_out</code> using its <code>.create_msg()</code>. Mutate it via capnp-generated setters. Send it via <code>.send()</code> or <code>.*_request()</code>. Receive it on the other side as a <code>Msg_in</code> (<code>struc::Msg_in</code>) and access it via capnp-generated getters. This may well be perfectly sufficient for many use cases: Messaging is a common IPC paradigm, and in the described capacity <code>struc::Channel</code> provides, on top of the basics: structured data via capnp schemas; and basic request-response/multiplex-demultiplex APIs.</p>
<p >However the abilities of the <a class="el" href="namespaceipc_1_1transport.html" title="Flow-IPC module providing transmission of structured messages and/or low-level blobs (and more) betwe...">ipc::transport</a> <b>structured layer</b> go beyond that basic and effective paradigm. Hand-wavily speaking, an <a class="el" href="classipc_1_1transport_1_1struc_1_1Msg__out.html" title="A structured out-message suitable to be sent via struc::Channel::send() (et al).">ipc::transport::struc::Msg_out</a> (which represents a <b>message</b>, as opposed to message instance, and is not so different conceptually from a container) can be seen as not a short-lived message &ndash; that exists essentially just before sending and just after receiving &ndash; but as a <em>data structure</em> whose lifetime is practically unlimited (if so desired).</p>
<p >It is probably clear already that capnp provides the ability to express many data structures (as ~anything can be built on top of <code>struct</code>s, <code>union</code>s, and <code>List</code>s). (Granted, things like sorted trees and hash-tables would need some some added code to be conveniently accessed directly within a capnp schema, but that is also possible.)</p>
<dl class="section note"><dt>Note</dt><dd>If a native capnp schema is insufficient for your data structure's performance or semantic needs, we provide first-class-citizen support for direct C++ data structures, including STL-compliant containers, in shared memory. See <a class="el" href="transport_shm.html">Shared Memory: Direct Allocation, Transport</a>.</dd></dl>
<p>However, beyond that, a number of capabilities are necessary to treat <code>struc::Msg_out</code> and the associated <b>message instances</b> (<a class="el" href="classipc_1_1transport_1_1struc_1_1Msg__in.html" title="A structured in-message instance suitable as received and emittable (to user) by struc::Channel.">ipc::transport::struc::Msg_in</a> a/k/a <a class="el" href="classipc_1_1transport_1_1struc_1_1Channel.html#a05d1cd2fc6a81a39c50a24322fb5c165" title="Encapsulation of any in-message instance received by a *this of this type.">ipc::transport::struc::Channel::Msg_in</a> upon receipt) as long-lived data structures shared among processes. The most basic such capabilities are:</p><ul>
<li>modifying a <code>Msg_out</code> after <code>.send()</code>ing (et al);</li>
<li>re-<code>.send()</code>ing (et al) a <code>Msg_out</code> over the same channel;</li>
<li>re-<code>.send()</code>ing (et al) a <code>Msg_out</code> over a different channel.</li>
</ul>
<p >As you will soon see, all of these are quite simple and in fact don't even involve any more APIs than those explained in <a class="el" href="chan_struct.html">Structured Message Transport</a>.</p>
<p >Potentially one might also want the following property:</p><ul>
<li>if a change is made to the original <code>Msg_out</code>, the change is immediately reflected in an associated <code>Msg_in</code> already received by a process (subject to synchronization that is the user's responsibility).</li>
</ul>
<p >As you will soon see, this merely requires the use of a SHM-backed serializer &ndash; but that is already assumed at least in the default recommendation and example code in <a class="el" href="chan_struct.html">Structured Message Transport</a>.</p>
<p >Finally, there is the matter of the the <b>lifetime</b> of a given <code>Msg_out</code> and associated <code>Msg_in</code>s. There are a few ways to think about this, but supposing one uses the <a class="el" href="namespaceipc_1_1session.html" title="Flow-IPC module providing the broad lifecycle and shared-resource organization – via the session conc...">ipc::session</a> paradigm for channel opening (and possibly SHM use), it can be roughly described as the following capabilities:</p><ul>
<li>a <code>Msg_out</code> (+ <code>Msg_in</code>s) lifetime that is at least equal to that of a particular <a class="el" href="classipc_1_1session_1_1Session.html" title="A documentation-only concept defining the local side of an IPC conversation (session) with another en...">ipc::session::Session</a>;</li>
<li>a lifetime exceeding that of the session's.</li>
</ul>
<dl class="section note"><dt>Note</dt><dd>For completeness we should mention that to be usable 100% completely as simply a data structure coequally shared among all relevant processes, a particular <code>Msg_*</code> would also need to be modifiable (via capnp-generated mutating API) on the receiver side. As of this writing Flow-IPC does not offer such an API. However this would be an incremental and not particularly difficult addition to Flow-IPC. We may add this in the foreseeable future. (Exception: The SHM-provider ipc::shm::arena_lend (SHM-jemalloc) is such that it is conceptually impossible to offer this feature in its case. However <a class="el" href="namespaceipc_1_1shm_1_1classic.html" title="ipc::shm sub-module with the SHM-classic SHM-provider. See ipc::shm doc header for introduction.">ipc::shm::classic</a> is fine in this respect.)</dd></dl>
<p>Advice: The topics in this page are not difficult, as long as one simply adjusts their understanding of what a <code>struc::Msg_out</code> (+ associated message instance objects <code>struc::Msg_in</code>) really is. The main adjustment: despite the existence of the convenience method <a class="el" href="classipc_1_1transport_1_1struc_1_1Channel.html#a394e0f9cfa5f9bd4ba9ee17d3d54a8b3" title="Creates/returns empty out-message, optionally also holding a native handle, to mutate and later send(...">ipc::transport::struc::Channel::create_msg()</a>, and the <code>struc::Channel</code> constructor arguments that make that work, a message is not in reality in any way "attached" to a particular channel. It is actually <em>an independent data structure</em> &ndash; in a way like a container. Once one understands its lifetime and this orthogonality to channels, it all (we feel) makes straightforward sense.</p>
<h3>A word on <code>Native_handle</code> transmission</h3>
<p >We've already mentioned this in <a class="el" href="chan_struct.html">Structured Message Transport</a> but to recap/expand: One can load a native handle (FD in POSIX/Unix/Linux parlance) into a <code>Msg_out</code>. A received <code>Msg_in</code> will then contain a handle referring to the same resource description, accessible via <a class="el" href="classipc_1_1transport_1_1struc_1_1Msg__in.html#aba0b83c9d8a7e8f782202406ca387e2c" title="The Native_handle – potentially null meaning none – embedded in this message.">ipc::transport::struc::Msg_in::native_handle_or_null()</a>. The <code>Msg_out</code>-stored <code>Native_handle</code> can be unloaded or replaced with another one; this has no effect on the associated <code>Msg_in</code>s. (The underlying resource, such as a file handle or network socket, will be freed once <em>all</em> handles proliferated this way have been freed. <code>Msg_out</code> will do that to its handle on destruction or replacement/unloading via <code>.store_native_handle_or_null()</code>. <code>Msg_in</code>-stored such handles are not auto-freed; it is the user's responsibility upon receipt to access via <code>.native_handle_or_null()</code> and further handle it as she will.)</p>
<p >Fundamentally these <code>Native_handle</code>s should be viewed as light-weight; in at least POSIX/Unix/Linux they are <code>int</code>s. Furthermore, as shown above, conceptually (not literally &ndash; it is not a mere <code>int</code> copy when crossing a process boundary) speaking they are always <em>copied</em> on transmission. In the rest of the page we speak at length about message payloads being copied or not-copied (zero-copy). To be clear, when we talk of this, we are speaking about the structured-data payload <em>only</em>. <code>Native_handle</code>s (conceptually) are always copied when transmitted.</p>
<h2>Re-sending a <code>Msg_out</code>; modifying a <code>Msg_out</code> </h2>
<p >These actions are so simple that we felt it best to just give recipes for them here at the top, without even first philosophizing about message lifetimes and the like.</p>
<p >To send a <code>Msg_out</code>, you already know you can use <code>struc::Channel</code> methods <code>.send()</code>, <code>async_request()</code>, <code>sync_request()</code>. Obviously up to that point you already know you can mutate the payload via capnp mutating API accessible via <code>struc::Msg_out::body_root()-&gt;</code>.</p>
<p >To modify it <em>after</em> a send: You just... do it. <code>body_root()-&gt;</code>, etc. Any <code>Builder</code>s you've saved from before the send will work just fine too (and be indistinguishable from those re-obtained via a re-invocation of <code>body_root()</code>.)</p>
<p >To re-send a <code>Msg_out</code>, whether you've modified it or not: You just... do it. Use the same <code>Msg_out</code>. (There is no way to copy it, as of this writing, so.... Though, technically, you could do that yourself via capnp fanciness; but I digress.)</p>
<p >All of that said, simple as it is, it doesn't answer questions that naturally arise; for example, if one modifies a <code>Msg_out</code>, does an existing related-<code>Msg_in</code> observer "see" it? Time to discuss all that.</p>
<hr  />
<dl class="section user"><dt>Send 1 message over 2+ different channels, with this one weird trick</dt><dd>Actually there is no trick. Got a <code>Msg_out</code>? You can send it over any <code>struc::Channel</code> you've got, as long as the template parameters match (most notably <code>Message_body</code>, the schema); or it won't compile. It doesn't matter which one's <code>.create_msg()</code> you used, or whether you used one at all (it can be <a class="el" href="chan_struct_advanced.html#msg_ct">explicitly constructed instead</a>).</dd></dl>
<dl class="section user"><dt></dt><dd>However this opens up further questions; like "Is it safe?" (Laurence Olivier -&gt; Dustin Hoffman). The answer is slightly tricky. Hopefully we will reveal all in the next couple of sections of this Manual page.</dd></dl>
<hr  />
<h2>Messages, message instances, message lifetime </h2>
<p >A <b>message</b> begins existence when an <a class="el" href="classipc_1_1transport_1_1struc_1_1Msg__out.html" title="A structured out-message suitable to be sent via struc::Channel::send() (et al).">ipc::transport::struc::Msg_out</a> (a/k/a <code>struc::Channel::Msg_out</code>) is constructed. A <b>message instance</b> begins existence <em>when it is sent via a <code>struc::Channel</code></em> via a send-method of thereof. It is never accessible, per se, by the user in the sender process; it is first accessible by a user upon receipt via a <code>struc::Channel</code> API (<code>.sync_request()</code> return value, an <code>.expect_*()</code> handler, or an <code>.async_request()</code> handler). Specifically, that message instance lives in that <code>struc::Msg_in</code> (a/k/a <code>struc::Channel::Msg_in</code>).</p>
<p >So the <code>Msg_out</code> is the message; the <code>Msg_in</code>s are the message instances <em>as they are sent+received, each time that occurs</em>.</p>
<p >The key idea to understand is the relationship between the message (<code>Msg_out</code>) and its 0+ message instances &ndash; <code>Msg_in</code>s. Simply put:</p><ul>
<li>A message instance is a read-only view of the message. (Per the "Note" above the "read-only" restriction may be lifted, in most cases, in an Flow-IPC in the foreseeable future.)</li>
<li>If the message is <b>SHM-backed</b>: A message instance accesses the same location in memory, directly, as the message. Perf-wise: no copy is made when the instance is created (i.e., it is sent/received). Algorithmic possibilities-wise: the message instance <em>is</em> the same data structure as the message.</li>
<li>If the message is <b>heap-backed</b>: A message instance accesses a different location in memory than the message. Perf-wise: a copy is made when the instance is created (again, at send/receive time). Algorithmic possibilities-wise: the message instance has the same structured as the message but is a <em>copy</em> thereof.</li>
</ul>
<p >We'll explain "SHM-backed" and "heap-backed" shortly.</p>
<h3>Message lifetime</h3>
<p >A message's <b>lifetime</b> ends at a certain time depending on whether it's SHM-backed or heap-backed. Ditto for message instance lifetime.</p>
<p >For heap-backed messages it is straightforward, as there's the message, and the instances are subsequent copies thereof.</p><ul>
<li>A heap-backed <em>message</em> lasts as long as the process in which it was allocated. One can even shut down all IPC sessions: it is still accessible, modifiable, everything. (This could be useful in its own right.) Saved capnp-generated <code>Builder</code>s continue working equally well and so on.</li>
<li>A heap-backed message <em>instance</em> lasts as long as the process in which its <code>Msg_in</code> was initially received-into (hence allocated, in the bowels of a <code>struc::Channel</code>). It's a copy of the original message: it's just (as of this writing) a read-only copy. Just as with the original message, you can end all IPC, and you can still access the message instance just fine including via saved capnp-generated <code>Reader</code>s.</li>
</ul>
<p >For SHM-backed messages it is more fun, as the message and subsequent message instances refer to the same data in RAM. The lifetime of those underlying data ends in the last destructor to complete among:</p><ul>
<li>the destructor of the original <code>struc::Msg_out</code>;</li>
<li>the destructor of each subsequently created <code>struc::Msg_in</code>.</li>
</ul>
<p >Thus, conceptually, there is a ref-count: 1 for the <code>Msg_out</code>; 1 for each <code>Msg_in</code>. Once it reaches zero, the lifetime of the data ends, and the RAM resources are returned for use by other items. (This is all thread-safe in cross-process fashion. It's fine if destructors run concurrently to each other, with a new related <code>Msg_in</code> being created in another process, etc.)</p>
<dl class="section note"><dt>Note</dt><dd><code>Msg_in</code>s are exclusively trafficked via <code>Msg_in_ptr</code>s which are mere <code>shared_ptr&lt;Msg_in&gt;</code>s; hence once a particular <code>shared_ptr</code> group reaches ref-count-zero, the destructor is invoked, and voilà for that particular <em>one</em> message instance. This applies regardless of SHM-backed versus heap-backed messages. Meanwile <code>Msg_out</code> can live wherever you want (you can wrap it in a <code>shared_ptr</code> if you want); but don't confuse where these <em>objects</em> live as opposed to where the <em>data structure</em> lives (similarly to a regular <code>std::vector</code> potentially living on the stack but allocating its buffer elsewhere, typically heap but depending on the <code><a class="el" href="classAllocator.html">Allocator</a></code> used elsewhere).</dd></dl>
<p><a class="anchor" id="msg_ct"></a></p><h3>Constructing messages; reusing messages among different <code>struc::Channel</code>s</h3>
<p >We can now discuss freely how one creates a <code>Msg_out</code>. You already know about <code>struc::Channel::create_msg()</code>, but in our expanded context here &ndash; of infinite lifetimes and SHM-arenas, oh my &ndash; it is not quite sufficient and possibly somewhat confusing, as it appears to associate a <em>message</em> with a <em>channel</em>; but the two are (as we've stated earlier) actually orthogonal.</p>
<dl class="section note"><dt>Note</dt><dd>A message <em>instance</em> is created internally by <code>struc::Channel</code> at receipt time. By the time you get the <code>Msg_in_ptr</code> into your code, it's already in existence, and it will go out of existence when that <code>shared_ptr&lt;Msg_in&gt; = Msg_in_ptr</code>'s shared-pointer group reaches ref-count-zero.</dd></dl>
<p>Formally speaking the backing (SHM versus heap; plus config) of any given <code>Msg_out</code> is controlled via the formal concepts <a class="el" href="classipc_1_1transport_1_1struc_1_1Struct__builder.html" title="A documentation-only concept defining the behavior of an object capable of zero-copy-serializing,...">ipc::transport::struc::Struct_builder</a> and <a class="el" href="structipc_1_1transport_1_1struc_1_1Struct__builder_1_1Config.html" title="Copy-ctible, copy-assignable, default-ctible type – informally, cheaply copyable and likely an aggreg...">ipc::transport::struc::Struct_builder::Config</a> (and the deserialization counterparts <a class="el" href="classipc_1_1transport_1_1struc_1_1Struct__reader.html" title="A documentation-only concept that is, conceptually, roughly what capnp::MessageReader is to capnp::Me...">ipc::transport::struc::Struct_reader</a> and <a class="el" href="structipc_1_1transport_1_1struc_1_1Struct__reader_1_1Config.html" title="Analogous to Struct_builder::Config but for deserialization.">ipc::transport::struc::Struct_reader::Config</a>). You can read all about them and their impls &ndash; or even potentially how to create your own for truly advanced fanciness &ndash; by following those links into the Reference and going from there. (In that case you will also need to understand <code>struc::Channel</code> non-tag constructor form as well as the related <code>Struct_builder_config</code> and <code>Struct_reader_config</code> class template paramers which match <code>Struct_builder::Config</code> and <code>Struct_reader::Config</code> concepts repsectively.) Here in the guided Manual we won't get into it to that level of formality and depth. We strive to keep it immediately useful but nevertheless sufficiently advanced for most needs.</p>
<p >So here are the relevant recipes with all currently available types of message backing. Let's start with the simplest one: <b>heap-backed messages</b>.</p>
<hr  />
<dl class="section user"><dt>When to choose heap-backing (over SHM-backing)?</dt><dd>As explained above transmitting a heap-backed message means a copy is made. This has the obvious undesirable performance hit: a copy is made at least into the transport and another out of the transport (possibly another inside the transport, depending on the transport) &ndash; instead of <em>zero</em> copies end-to-end. Furthermore this limits the size of a capnp data structure's <em>leaf</em> to what can be serialized into one unstructured message bearable by the underlying <a class="el" href="classipc_1_1transport_1_1Channel.html" title="Peer to a bundle of 1-2 full-duplex pipe(s), one for transmitting unstructured binary blobs; the othe...">ipc::transport::Channel</a> transport(s). (For detail of this limitation see <a class="el" href="classipc_1_1transport_1_1struc_1_1Heap__fixed__builder.html" title="Implements Struct_builder concept by straightforwardly allocating fixed-size segments on-demand in th...">ipc::transport::struc::Heap_fixed_builder</a> class doc header. It also expands on the present discussion generally.) However it may well still be useful for many applications. Informally we recommend SHM-backing as the default choice; but certainly there are use cases where heap-backing works better due to its simplicity. Internally we use it to <em>implement</em> certain internal aspects of the SHM-jemalloc SHM-provider (for example).</dd></dl>
<hr  />
<p >For SHM-backed message transmission: Use <a class="el" href="namespaceipc_1_1transport_1_1struc.html#aca041b0efd4a9ae18e10088b2ee3c052" title="Convenience alias: Use this when constructing a struc::Channel with tag Channel_base::S_SERIALIZE_VIA...">ipc::transport::struc::Channel_via_heap</a> as your <code>Structured_channel_t</code> template in place of what we used in the example in <a class="el" href="chan_struct.html">Structured Message Transport</a>. There we used <a class="el" href="namespaceipc_1_1transport_1_1struc_1_1shm_1_1classic.html#ac543b500a85f556564e592cbbefb6185" title="Convenience alias: Use this when constructing a struc::Channel with tag Channel_base::S_SERIALIZE_VIA...">ipc::transport::struc::shm::classic::Channel</a> albeit invisibly via <code>Session::Structured_channel</code> alias. If your chosen <code>Session</code> type is <em>not</em> SHM-backed &ndash; meaning you will not be using <code><a class="el" href="namespaceipc_1_1session.html" title="Flow-IPC module providing the broad lifecycle and shared-resource organization – via the session conc...">ipc::session</a></code>-provided SHM capabilities at all and ultimately thus chose to use <a class="el" href="namespaceipc_1_1session.html#a77d8fda00e1e17f595a6a5d4c44375e7" title="A vanilla Client_session with no optional capabilities.">ipc::session::Client_session</a> or <a class="el" href="classipc_1_1session_1_1Session__server.html" title="To be instantiated typically once in a given process, an object of this type asynchronously listens f...">ipc::session::Session_server</a> as opposed to a SHM-backed variant thereof &ndash; then <code>Session::Structured_channel</code> will yield <a class="el" href="namespaceipc_1_1transport_1_1struc.html#aca041b0efd4a9ae18e10088b2ee3c052" title="Convenience alias: Use this when constructing a struc::Channel with tag Channel_base::S_SERIALIZE_VIA...">ipc::transport::struc::Channel_via_heap</a>. Otherwise you can use <a class="el" href="classipc_1_1session_1_1Session__mv.html#ac205819d4a3a066f31b5af311ff40a2a" title="Implements Session API per contract.">ipc::session::Session_mv::Structured_channel</a> alias; or <a class="el" href="namespaceipc_1_1transport_1_1struc.html#aca041b0efd4a9ae18e10088b2ee3c052" title="Convenience alias: Use this when constructing a struc::Channel with tag Channel_base::S_SERIALIZE_VIA...">ipc::transport::struc::Channel_via_heap</a> explicitly.</p>
<p >There is a number of ways to construct a heap-backed <code>Msg_out</code>. Assuming you use <a class="el" href="namespaceipc_1_1session.html" title="Flow-IPC module providing the broad lifecycle and shared-resource organization – via the session conc...">ipc::session</a> to obtain channels, which you generally should barring a very good reason not to, then <code>.create_msg()</code> will work most easily. The resulting <code>Msg_out</code> (and any resulting <code>Msg_in</code>s) are transmissible over that channel or any other <code>struc::Channel</code> upgraded-from a <code>Channel</code> opened from the same <code>Session</code> or <code>Session</code> of the same concrete C++ type. The latter includes (on session-server side) any <code>Session</code> arising from the same <code>Session_server</code> (of which there should be exactly 1 per process).</p>
<p >Messages are conceptually decoupled from specific channels, and your program may be structured in such a way as to not have a <code>struc::Channel</code> available easily when constructing the <code>Msg_out</code>. In that case you can easily construct a matching-type <code>struc::Msg_out</code> explicitly without <code>.create_msg()</code>. You will need a <code>Struct_builder::Config</code> object (which is a light-weight item). This is obtained easily via <a class="el" href="classipc_1_1session_1_1Session__mv.html#a107684fdd5d233549c6daac26a81a542" title="Utility that obtains a heap-based (non-zero-copy) Struct_builder::Config, constructed with the most e...">ipc::session::Session_mv::heap_fixed_builder_config()</a> including the <code>static</code> overload that merely requires a <code>Logger*</code> (which can itself be null if desired).</p>
<p >Going outside the <a class="el" href="namespaceipc_1_1session.html" title="Flow-IPC module providing the broad lifecycle and shared-resource organization – via the session conc...">ipc::session</a> paradigm is (as earlier forewarned) outside our scope here, but briefly we can explain that if you go that way you can construct the <code>Struct_builder::Config</code> &ndash; actually concretely an <a class="el" href="structipc_1_1transport_1_1struc_1_1Heap__fixed__builder_1_1Config.html" title="Implements Struct_builder::Config sub-concept.">ipc::transport::struc::Heap_fixed_builder::Config</a> &ndash; explicitly. Certain values will be required for <em>that</em> constructor, and they must be such that all <code>Channel</code>s over which the message shall be transmitted (recall that with heap-backed messages the actual message's serialization is copied into/out of the transport) shall be capable of transmitting even the largest possible capnp-serialization segment that would ever be generated given your data. Again: this is outside our scope here, but the Reference includes all necessary information for this advanced use case.</p>
<p >Now let's talk about constructing <b>SHM-backed messages</b>.</p>
<p >In fact, in <a class="el" href="chan_struct.html">Structured Message Transport</a>, we provided the recipe for a common use case; namely:</p><ul>
<li>Scope: session-scope (specified by <a class="el" href="classipc_1_1transport_1_1struc_1_1Channel__base.html#a9bdb94c1ba4fac92fbbd0f843e8525ba" title="The sole value of the tag type Serialize_via_session_shm.">ipc::transport::struc::Channel_base::S_SERIALIZE_VIA_SESSION_SHM</a> tag arg to <code>struc::Channel</code> constructor);</li>
<li>SHM-provider: SHM-classic (<a class="el" href="namespaceipc_1_1shm_1_1classic.html" title="ipc::shm sub-module with the SHM-classic SHM-provider. See ipc::shm doc header for introduction.">ipc::shm::classic</a> &ndash; specified via the compile-time <code>struc::Channel</code> type selected, namely in our example code <code>Session::Structured_channel</code> which resolved to <a class="el" href="namespaceipc_1_1transport_1_1struc_1_1shm_1_1classic.html#ac543b500a85f556564e592cbbefb6185" title="Convenience alias: Use this when constructing a struc::Channel with tag Channel_base::S_SERIALIZE_VIA...">ipc::transport::struc::shm::classic::Channel</a> due to the concrete type of <code>Session</code>).</li>
</ul>
<p >What does session-scope mean? Briefly it means that the lifetime of any resulting messages is limited to that of the <code>Session</code> given by you as an argument to the <code>struc::Channel</code> constructor. The alternative is app-scope. (Please read <a href="./session_app_org.html#scope">elsewhere in the Manual</a> to understand these 2 alternatives.) In many applications session-scope is sufficient.</p>
<p >If indeed session-scope is sufficient, then you can use SHM-classic SHM-provider as above; or else select SHM-jemalloc. If writing code in the fashion recommended in this Manual this will flow automatically from your original choice of <code>Client_session</code> (<a class="el" href="namespaceipc_1_1session_1_1shm_1_1classic.html#ac6c443a72e0f22710c83bb1e8c229618" title="This is to session::Client_session what shm::classic::Server_session is to session::Server_session.">ipc::session::shm::classic::Client_session</a> versus <a class="el" href="namespaceipc_1_1session_1_1shm_1_1arena__lend_1_1jemalloc.html#a837b4566cb2a734eb0ba58f285d084fe" title="This is to session::Client_session what shm::arena_lend::jemalloc::Server_session is to session::Serv...">ipc::session::shm::arena_lend::jemalloc::Client_session</a>) or <code>Session_server</code> (ditto modulo the last identifier name). Otherwise use <a class="el" href="namespaceipc_1_1transport_1_1struc_1_1shm_1_1arena__lend_1_1jemalloc.html#ab68a69ad42d97d754f4e77c8cadd3a5e" title="Convenience alias: Use this when constructing a struc::Channel with tag Channel_base::S_SERIALIZE_VIA...">ipc::transport::struc::shm::arena_lend::jemalloc::Channel</a>.</p>
<p >If app-scope is required, then:</p><ul>
<li>Use <a class="el" href="classipc_1_1transport_1_1struc_1_1Channel__base.html#af732121649271d1368a2fb637917ce7b" title="The sole value of the tag type Serialize_via_app_shm.">ipc::transport::struc::Channel_base::S_SERIALIZE_VIA_APP_SHM</a> tag arg to <code>struc::Channel</code> constructor.</li>
<li>You can still select either SHM-classic or SHM-jemalloc.<ul>
<li>However, in the latter case (and the latter case only): The <code>session</code> argument to that constructor must be a <code>Server_session</code>; trying to use a <code>Client_session</code> will result in a compile error. With SHM-jemalloc, on the session-client side, it is only possible to construct session-scope SHM-backed data. This is due to the nature of client-sessions combined with the nature of arena-lend style of SHM-provider.</li>
</ul>
</li>
</ul>
<p >Which SHM-provider to use? That is a major decision and affects not just SHM-backed messages but direct-storage of C++ structures in SHM; see <a class="el" href="transport_shm.html">Shared Memory: Direct Allocation, Transport</a> w/r/t the latter. Without a deep dive: See a <a href="./safety_perms.html#shm_safety">safety-oriented discussion</a> and <a href="./safety_perms.html#shm_safety_other_considerations">nearby recap of non-safety considerations</a>. A lengthier description can be <a class="el" href="namespaceipc_1_1shm.html">found in the Reference</a>.</p>
<p >Having constructed the properly-typed SHM-backed-config <code>struc::Channel</code> properly using these knobs, it is easiest to use <code>.create_msg()</code> to create <code>Msg_out</code>. Any allocation required as you subsequently mutate it shall occur in the proper SHM arena, and resulting <code>Msg_in</code>s on the receiving side shall properly access it directly there.</p>
<p >Messages are conceptually decoupled from specific channels, and your program may be structured in such a way as to not have a <code>struc::Channel</code> available easily when constructing the <code>Msg_out</code>. In that case you can easily construct a matching-type <code>struc::Msg_out</code> explicitly without <code>.create_msg()</code>. You will need a <code>Struct_builder::Config</code> object (which is a light-weight item). This is obtained via <a class="el" href="classipc_1_1session_1_1shm_1_1classic_1_1Session__mv.html#a85bb0b948b7b54c06557edd3d03d66bd" title="Returns builder config suitable for capnp-serializing out-messages in SHM arena session_shm().">ipc::session::shm::classic::Session_mv::session_shm_builder_config()</a> (session-scope, SHM-classic) or <a class="el" href="classipc_1_1session_1_1shm_1_1classic_1_1Session__mv.html#a0fbea0563835e852069cdaab506dd92c" title="Identical to session_shm_builder_config() but backed by SHM arena app_shm() instead of session_shm().">ipc::session::shm::classic::Session_mv::app_shm_builder_config()</a> (app-scope, SHM-classic). For SHM-jemalloc substitute <a class="el" href="classipc_1_1session_1_1shm_1_1arena__lend_1_1jemalloc_1_1Session__mv.html#ad00593f745b49c41fa446bdf4250f2e9" title="Returns builder config suitable for capnp-serializing out-messages in SHM arena session_shm().">ipc::session::shm::arena_lend::jemalloc::Session_mv::session_shm_builder_config()</a> or <a class="el" href="classipc_1_1session_1_1shm_1_1arena__lend_1_1jemalloc_1_1Server__session.html#a26c1d34c7be1002cc9fc8ae9eeae8164" title="Returns builder config suitable for capnp-serializing out-messages in SHM arena app_shm().">ipc::session::shm::arena_lend::jemalloc::Server_session::app_shm_builder_config()</a>. All four methods are available via your <code>Session</code> object (e.g., <code>session.session_shm_builder_config()</code>). The 2 app-scope methods have counterparts <a class="el" href="classipc_1_1session_1_1shm_1_1classic_1_1Session__server.html#ad0dafea47293d23e079b6b5c5c6a9cef" title="Returns builder config suitable for capnp-serializing out-messages in SHM arena app_shm() for the sam...">ipc::session::shm::classic::Session_server::app_shm_builder_config()</a> and <a class="el" href="classipc_1_1session_1_1shm_1_1arena__lend_1_1jemalloc_1_1Session__server.html#aa548c3379d4b8a8f0b9d618b0c1da7ba" title="Returns builder config suitable for capnp-serializing out-messages in SHM arena app_shm() for the sam...">ipc::session::shm::arena_lend::jemalloc::Session_server::app_shm_builder_config()</a> in case an individual <code>Session</code> is not easily accessible.</p>
<p >Going outside the <a class="el" href="namespaceipc_1_1session.html" title="Flow-IPC module providing the broad lifecycle and shared-resource organization – via the session conc...">ipc::session</a> paradigm is (as earlier forewarned) outside our scope here, but briefly we can explain that, if you go that way, you can explicitly construct the <code>Struct_builder::Config</code> &ndash; actually concretely an <code><a class="el" href="structipc_1_1transport_1_1struc_1_1shm_1_1Builder_1_1Config.html" title="Implements Struct_builder::Config sub-concept.">ipc::transport::struc::shm::Builder::Config</a>&lt;A&gt;</code> (where <code>A</code> is SHM-provider-determined <code>Shm_arena</code> type, namely <a class="el" href="classipc_1_1shm_1_1classic_1_1Pool__arena.html" title="A SHM-classic interface around a single SHM pool with allocation-algorithm services by boost....">ipc::shm::classic::Pool_arena</a> or ipc::shm::arena_lend::jemalloc::Ipc_arena. The <code>Config</code> can, basically, take zeroes for the framing integers and, crucially, a pointer to the <code>Shm_arena</code> in which to allocate. Details as to where to get that <code>Shm_arena</code> are well beyond our scope here; but if you're foregoing <a class="el" href="namespaceipc_1_1session.html" title="Flow-IPC module providing the broad lifecycle and shared-resource organization – via the session conc...">ipc::session</a>, you presumably know what you're doing in the first place in this regard.</p>
<p >The last, and considerably important, subject to discuss about SHM-backed <code>Msg_out</code>s is: Which channels can safely/properly transmit a given <code>Msg_out</code>, once it has been constructed and filled out via mutation?</p>
<p >Let's take the session-scope <code>Msg_out</code>s, the simpler case, first. The answer: regardless of what API was used to construct one, it can be sent over any <code>struc::Channel</code> upgraded-from a <code>Channel</code> opened via the same <a class="el" href="classipc_1_1session_1_1Session.html" title="A documentation-only concept defining the local side of an IPC conversation (session) with another en...">ipc::session::Session</a> as the one SHM-backing the original <code>Msg_out</code>. What <code>Session</code> is that? Answer: If you used <code>.create_msg()</code>, then it's the one passed-in, by you, to <code>struc::Channel</code> constructor. If you used explicit <code>struc::Msg_out</code> constructor, then the backing session is the one you used when calling <code>session.session_shm_builder_config()</code> to provide the required <code>Struct_builder::Config</code>.</p>
<p >So the simplest use case is simply using <code>x.create_msg()</code> and then sending it (as many times as desired) via <code>x.send()</code> (et al); but instead of <code>x.send()</code> (et al) you can use various values instead of <code>x</code> &ndash; as long as it's from the same session as <code>x</code>. Also &ndash; as channels are otherwise decoupled from their source sessions &ndash; ensure the source session is still open. (If you followed advice in <a class="el" href="session_app_org.html">Sessions: Teardown; Organizing Your Code</a>, you will be good to go. You would have ceased to use a channel upon learning of the closure of its source session; that is assuming you didn't yourself close them both in the first place.)</p>
<p >If not using <code>x.create_msg()</code> but direct <code>Msg_out</code> construction, you're still fine: Just make sure you keep a clean design as to which session corresponds to what channel(s) and message(s).</p>
<p >As to app-scope <code>Msg_out</code>s: On the session-client side, with SHM-jemalloc there are no app-scope <code>Msg_out</code>s, so it's a moot question. On the session-client side, with SHM-classic, however, there are app-scope <code>Msg_out</code>s... but it is improper to let a <code>Msg_out</code> survive past the end of a given client-side session. <em>However</em> resulting <code>Msg_in</code>s (on the opposing side, so the session-server side) will survive just fine &ndash; until the entire <code>Session_server</code> is destroyed; which would occur only if one intends to end all relevant IPC work in the process. The ability to make client-created messages survive past a given session is unique to SHM-classic over SHM-jemalloc. (It is easy to conceive of this being useful in a server-side cache application.)</p>
<p >App-scope <code>Msg_out</code>s on the session-server side can be transmitted over any channel from <em>any</em> session from the same <code>Session_server</code> &ndash; of which, again, there shall be exactly 1 per process &ndash; as long as the <code>Session_server</code> object has not been destroyed.</p>
<p >The next page is: <a class="el" href="transport_shm.html">Shared Memory: Direct Allocation, Transport</a>.</p>
<hr  />
<center><b>MANUAL NAVIGATION:</b> <a class="el" href="chan_struct.html">Preceding Page</a> - <a class="el" href="transport_shm.html">Next Page</a> - <a href="./pages.html"><b>Table of Contents</b></a> - <a class="el" href="namespaceipc.html"><b>Reference</b></a></center> </div></div><!-- contents -->
</div><!-- PageDoc -->
<!-- start footer part -->
<hr class="footer"/><address class="footer"><small>
Generated on Thu May 2 2024 23:56:38 for Flow-IPC by&#160;<a href="https://www.doxygen.org/index.html"><img class="footer" src="doxygen.svg" width="104" height="31" alt="doxygen"/></a> 1.9.4
</small></address>
</body>
</html>
